51 research outputs found
GRACE: Online Gesture Recognition for Autonomous Camera-Motion Enhancement in Robot-Assisted Surgery
Camera navigation in minimally invasive surgery changed significantly since the introduction of robotic assistance. Robotic surgeons are subjected to a cognitive workload increase due to the asynchronous control over tools and camera, which also leads to interruptions in the workflow. Camera motion automation has been addressed as a possible solution, but still lacks situation awareness. We propose an online surgical Gesture Recognition for Autonomous Camera-motion Enhancement (GRACE) system to introduce situation awareness in autonomous camera navigation. A recurrent neural network is used in combination with a tool tracking system to offer gesture-specific camera motion during a robotic-assisted suturing task. GRACE was integrated with a research version of the da Vinci surgical system and a user study (involving 10 participants) was performed to evaluate the benefits introduced by situation awareness in camera motion, both with respect to a state of the art autonomous system (S) and current clinical approach (P). Results show GRACE improving completion time by a median reduction of 18.9s (8.1% ) with respect to S and 65.1s (21.1% ) with respect to P. Also, workload reduction was confirmed by statistical difference in the NASA Task Load Index with respect to S (p < 0.05). Reduction of motion sickness, a common issue related to continuous camera motion of autonomous systems, was assessed by a post-experiment survey ( p < 0.01 )
Learning Deep Nets for Gravitational Dynamics with Unknown Disturbance through Physical Knowledge Distillation: Initial Feasibility Study
Learning high-performance deep neural networks for dynamic modeling of high
Degree-Of-Freedom (DOF) robots remains challenging due to the sampling
complexity. Typical unknown system disturbance caused by unmodeled dynamics
(such as internal compliance, cables) further exacerbates the problem. In this
paper, a novel framework characterized by both high data efficiency and
disturbance-adapting capability is proposed to address the problem of modeling
gravitational dynamics using deep nets in feedforward gravity compensation
control for high-DOF master manipulators with unknown disturbance. In
particular, Feedforward Deep Neural Networks (FDNNs) are learned from both
prior knowledge of an existing analytical model and observation of the robot
system by Knowledge Distillation (KD). Through extensive experiments in
high-DOF master manipulators with significant disturbance, we show that our
method surpasses a standard Learning-from-Scratch (LfS) approach in terms of
data efficiency and disturbance adaptation. Our initial feasibility study has
demonstrated the potential of outperforming the analytical teacher model as the
training data increases
Fully Immersive Virtual Reality for Skull-base Surgery: Surgical Training and Beyond
Purpose: A virtual reality (VR) system, where surgeons can practice
procedures on virtual anatomies, is a scalable and cost-effective alternative
to cadaveric training. The fully digitized virtual surgeries can also be used
to assess the surgeon's skills using measurements that are otherwise hard to
collect in reality. Thus, we present the Fully Immersive Virtual Reality System
(FIVRS) for skull-base surgery, which combines surgical simulation software
with a high-fidelity hardware setup.
Methods: FIVRS allows surgeons to follow normal clinical workflows inside the
VR environment. FIVRS uses advanced rendering designs and drilling algorithms
for realistic bone ablation. A head-mounted display with ergonomics similar to
that of surgical microscopes is used to improve immersiveness. Extensive
multi-modal data is recorded for post-analysis, including eye gaze, motion,
force, and video of the surgery. A user-friendly interface is also designed to
ease the learning curve of using FIVRS.
Results: We present results from a user study involving surgeons with various
levels of expertise. The preliminary data recorded by FIVRS differentiates
between participants with different levels of expertise, promising future
research on automatic skill assessment. Furthermore, informal feedback from the
study participants about the system's intuitiveness and immersiveness was
positive.
Conclusion: We present FIVRS, a fully immersive VR system for skull-base
surgery. FIVRS features a realistic software simulation coupled with modern
hardware for improved realism. The system is completely open-source and
provides feature-rich data in an industry-standard format.Comment: IPCAI/IJCARS 202
Improving Surgical Situational Awareness with Signed Distance Field: A Pilot Study in Virtual Reality
The introduction of image-guided surgical navigation (IGSN) has greatly
benefited technically demanding surgical procedures by providing real-time
support and guidance to the surgeon during surgery. \hi{To develop effective
IGSN, a careful selection of the surgical information and the medium to present
this information to the surgeon is needed. However, this is not a trivial task
due to the broad array of available options.} To address this problem, we have
developed an open-source library that facilitates the development of multimodal
navigation systems in a wide range of surgical procedures relying on medical
imaging data. To provide guidance, our system calculates the minimum distance
between the surgical instrument and the anatomy and then presents this
information to the user through different mechanisms. The real-time performance
of our approach is achieved by calculating Signed Distance Fields at
initialization from segmented anatomical volumes. Using this framework, we
developed a multimodal surgical navigation system to help surgeons navigate
anatomical variability in a skull base surgery simulation environment. Three
different feedback modalities were explored: visual, auditory, and haptic. To
evaluate the proposed system, a pilot user study was conducted in which four
clinicians performed mastoidectomy procedures with and without guidance. Each
condition was assessed using objective performance and subjective workload
metrics. This pilot user study showed improvements in procedural safety without
additional time or workload. These results demonstrate our pipeline's
successful use case in the context of mastoidectomy.Comment: First two authors contributed equally. 6 page
An Open-Source Research Kit for the da Vinci ® Surgical System
Abstract-We present a telerobotics research platform that provides complete access to all levels of control via opensource electronics and software. The electronics employs an FPGA to enable a centralized computation and distributed I/O architecture in which all control computations are implemented in a familiar development environment (Linux PC) and lowlatency I/O is performed over an IEEE-1394a (FireWire) bus at speeds up to 400 Mbits/sec. The mechanical components are obtained from retired first-generation da Vinci R Surgical Systems. This system is currently installed at 11 research institutions, with additional installations underway, thereby creating a research community around a common open-source hardware and software platform
- …